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Abstract 

Given a probability distribution over a set of n words to be transmitted, the Huffman Coding problem 
is to find a minimal-cost prefix free code for transmitting those words. The basic Huffman coding problem 
can be solved in 0(n log n) time but variations are more difficult. One of the standard techniques for 
solving these variations utilizes a top-down dynamic programming approach. 

In this paper we show that this approach is amenable to dynamic programming speedup techniques, 
permitting a speedup of an order of magnitude for many algorithms in the literature for such variations 
as mixed radix, reserved length and one-ended coding. These speedups are immediate implications of a 
general structural property that permits batching together the calculation of many DP entries. 
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1 Introduction 



Optimal prcfix-frcc coding, or Huffman coding, is a standard compression technique. Consider an encoding 
alphabet T, ~ {cti, . . . , Or). A code W = {'Wi,'W2, ■ ■ ■ , Wn} is a set of code words Wi £ S*. Code W is prefix- 
free if Vw, w' & W w is not a prefix of w' . As an example, {01, 00, 100} is a prefix-free code but {01, 00, 001} 
is not, because 00 is a prefix of 001. 

For w G Tj*, let jwl devote the length of w, i.e., the number of characters in w. For example |0101| = 4. 
The input to the problem is a discrete probability distribution P = {pi,p2, ■ ■ ■ ,Pn}, J2iPi = 1) Pi ^ 0- 
The output is a a prefix-free code W = {wi,W2, ■ ■ ■ ,Wn} whose expected encoding length Y17=iPi\'^i\ 
minimized over all n word prefix-free codes. Formally set Cost{W, P) = ^iPi- Then 

Cost{P)= min Cost{W',P) (1) 

W'CS*, \W'\ = n 

W is prefix-free 

In [16j . Huffman gave a classical 0(7ilogn) time greedy algorithm for solving the binary case (r = 2) 
of this problem. Huffman also extended the algorithm to solve the general r-ary case with the same time 
bound. If the pi's are given in sorted order, Huffman's algorithm can be improved to 0{rn) time |20] . 

The correctness of the Huffman algorithm, although easy to prove, is very strongly dependent upon 
properties of optimal prefix-free codes. Almost any extra constraint or generalization added to the problem 
description will invalidate the algorithm's correctness. Many such constraints/generalizations appear in the 
literature ([1] is a nice survey) and all require special purpose algorithms to address them. 

Some examples of such prefix-free coding problems are Length-limited coding e.g, |171ll8l [51ll9|. Unequal- 
cost coding e.g., [71 1151 [51 113j Mixed-radix coding [11| . Reserved-length coding [J, and One-ended coding 

Enano], 

The major observation is that all of the best algorithms known for these problems use some form of 
dynamic programming (DP) to build an optimal (min-cost) coding tree that corresponds to an optimal code. 

These DPs primarily differ in whether they build the tree from the bottom-up or the top-down. The 
best algorithms for Length-limited and Unequal-cost coding use what essentially reduces to a bottom-up DP 
model combined with some DP-spcedup techniques, e.g., Monge speedups using the SMAWK algorithm of 
[2] (see [8| for an example of this technique and [Ej for a more sophisticated but more specialized speedup 
method); the best algorithms for Reserved-length and One-ended coding use a top-down DP approach. 
(Mixed-radix coding [11] uses a totally different DP approach described later) 

Length-Limited and Unequal-Cost coding could be solved using a top-down approach but the bottom-up 
solutions are better for two reasons. The first is that the bottom-up solutions use a more compact solution 
space than corresponding top-down ones would. This is due to the exploitation of some very problem-specific 
combinatorial structures of their corresponding optimal code trees. The second is that their bottom-up DPs 
turn out to have special properties, e.g., the Monge property, which enable speeding up the calculation of 
table entries. The top-down DPs used in the last two problems don't have such a compact representations 
and they also, before this paper, didn't seem to possess any special property that would lead to speedups. 

The main result of this paper is a revisiting of the generic top-down DP approach for solving prefix-free 
coding problems. We will show that, in this setup, many natural coding problems will have an obvious and 
simple batching speedup. That is, we will be able to partition the DP table entries into smaller batches 
(groups) and exploit relationships between entries within a batch to fill in all of the entries in each batch 
in 0(1) amortized time per entry. This will enable speeding up known solutions to the last three problems 
by at least one factor of n. The interesting observation is that the same speedup technique works for all of 
these problems. Table [1] lists the speedups. 

1.1 The problems 

We start by quickly recalling the standard correspondence between prefix- free codes and trees. Let r = jSj 
be the size of the alphabet and consider an r-ary tree T in which the i^'^edge leaving a node is labelled with 
character ai. Associate with node v € T the unique word read off walking down the path from the root to 
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Problem 


Previous Best Result 


This paper 


Mixed Radix Coding 


0{n'^ log n) HU 




Reserved Length Coding: 
g specific lengths given 




0{gn^) 


Reserved Length Coding: 
at most g lengths allowed 


Oig^n^log^n) [4] 


Oign"^ logn) 


Onc-cndcd Coding 


0{n^) [10| 





Table 1: A comparison of our new results with previous ones 




VQ V7 

Code[ui] = 01 Code[i;5] = 101 
Code[i;2] = 11 Codeine] = 0000 
Code[i.3] = 001 Codejuy] = 0001 
Code[i;4] = 100 




0/1 \2 0/ 

"5 "(i "7 "8 ^'!J ^10 



Code[t;i] = 1 Codeine] = Oil 

Code[u2] = 3 Codelurl = 012 

Codejua] = 00 Code [us] =210 

Code[D4i = 20 Code[i.g] = 211 

Codejur,] = 010 Code[i.io] = 212 




ti4 ti5 vb V7 tig Vf) via till 



Code[t>i] = 1 Code[i;7] = 00011 

Code[ti2] = 001 Codefus] = 01100 

Code[t;3] = 010 Code[i;!)] = 01101 

Code[j;4] = 00000 Code[i;io] = OHIO 

Code[t>5] = 00001 Code[i;ii] = 01111 
Code[ti6] = 00010 



Figure 1: Examples of the code-tree correspondence. Codes are written below their corresponding tree. The 
leftmost figure is a standard binary tree. The middle is a Mixed-radix tree with level arities {to,ti,t2) = 
(4,2,3). The rightmost is a Reserved-length tree with codewords only on levels A = {1,3,5}. Note that 
leaves in all the trees are labelled from top-to bottom 



V. The set of words W associated with the leaves of T is prefix-free. Convcrsly, given a prefix-free code W 
one can build a tree whose leaves are exactly the nodes associated with the words of W. See Figure [1] 

Given tree T associated with code W, denote its leaves by vi,V2, ■ ■ ■ ,Vn where Vi is the leaf associated 
with word Wi. Let d(vi) be the depth of Vi in T. By the correspondence, d{vi) = \wi\ so 

Cost{T, P)^Y. = I^'l^'^ = Cost{W, P) 

i i 

where Cost{T, P) can be understood as the weighted external-path-length of T. So the prefix-free coding 
problem is equivalent to finding a tree with minimal external path length. For this reason, most algorithms 
for finding prefix-free codes are stated as tree algorithms. 

We now quickly discuss the problems mentioned in the previous sections and their tree equivalents and 
then state our new results for these problems. 

Mixed-Radix Coding: 

In Mixed-Radix Coding the size of the encoding alphabet used depends upon the position of the character 
within the codeword. This corresponds to constructing a tree in which the arity (number of children) of an 
internal node depends upon the level of the node. That is, as part of the problem definition, we are given a 
sequence tg, ti, . . . of integers, ti>2 such that the maximum arity of a node on level i is ti. 

The coding version of the problem was motivated [11] by coding with side-channel information and the 
tree version by problems in multi-level data storage. Chu and Gill [llj solved this problem by introducing 
an alphabetic version of it and then solving a special case of the alphabetic version. Their algorithm runs in 
O(n^logn) time; we will improve this to Oiji?). 

Reserved- Length Coding: 

Recall that \wi\ is the length of the i^'^codeword. In reserved-length coding there are specific restrictions as 
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to the permitted values of |mi[. There are two versions of this problem. In the first version, the given-lengths 
case, A = {71,72, . . . ,7g} is given as part of the input and we must find a minimum-cost code such that 
Vi, e A. This corresponds to building a min-cost tree in which all leaves are on levels in A. 

In the second version, the g-lengths case, A is not given in advance. The restriction now is to find a 
minimum-cost code under the restriction that |A| < g, where A is the set of codeword lengths used. This 
corresponds to building a tree in which at most g levels may contain leaves. 

Baer [4] introduces these problems in the context of fast decoding and used a top-down DP approach to 
solve the first one in 0(|A|n'') time and the second one in 0(7i'^.g'^ log^ n) time . We will reduce these two 
cases, respectively, to 0(|A|n^) and O(n^glogn) time. 
One-Ended Coding: 

In One-Ended coding the aim is to find a min-cost binary prefix-free code in which every word must end 
with a 1. This corresponds to finding a min-cost tree in which only right leaves (leaves that are the right 
children of their parents) are labelled with the pi and counted in the calculation of the cost. 

One-Ended Coding was introduced by Berger and Yeung [6] in the context of self-synchronizing codes. 
Their algorithm ran in exponential time. This was later improved by Dc Santis, CapoccUi and Persiano [9] 
to another exponential-time algorithm with a smaller exponential base. Chan and Colin [10] showed how to 
use top-down DP to derive an 0{n^) time algorithm. We will reduce this down to an 0{ii?) time one. 

In Section [2] we introduce a new coding problem called Generalized Mixed-Radix Coding, develop a top- 
down DP approach for solving it and then speed it up by batching. In Section[3]we reduce both Mixed-Radix 
Coding and Reduced Length Coding to (multiple) applications of CMR and thus take adavantage of the DP 
speedup. In Section [¥] we reduce the running time of One-Ended coding using an almost identical technique. 
Since the analysis of One-Ended coding is very similar to that of the CMR problem, we do not provide the 
details in this extended abstract (but they are available in the appendix). 

2 The Top Down DP for Generalized Mixed-Radix Coding 

We start by introducing the Ceneralized Mixed-Radix (CMR) problem, develop a top-down DP for solving 
it and then see how to speed it up. 

In a generalized mixed radix tree, both the arity of an internal node v and the length of an edge leaving 
V depends on the level of v. Figure [2] illustrates these and other definitions in this section. More formally. 

Definition 1 Given a a sequence of arities R = (ri, r2, . . .) and a sequence of edge length C = (ci, C2 . . .), 
a generalized mixed radix (CMR) tree T satisfying R, C is a tree in which internal node v at level i — 1 has 
at most ri children and and the length of an edge from v to any of its children is Ci . 

We now distinguish between the level i{v) of a node v, which is the number of edges from the root to v, 
and its depth, which is the weighted path length from the root to v. More formally. 

Definition 2 The level of node v in tree T is the number of edges on the unique path from the root to v and 
will be denoted by ({v). The level of tree T is 



The depth of a node on level i of the tree will be the sum of the lengths of the edges on the path from the 
root to level i, i.e., depth{v) — L{i) — X]j=i '^i- '^^^^ depth of tree T will be depth{T) ~ L{i{T)). 

There is an obvious definition of cost in such trees. 

Definition 3 Given R, C as above 

Let T be any generalized mixed radix tree T for R, C with n leaves labeled vi,V2, - ■ ■ ,Vn- Then 



The problem to be solved is, given P and R,C, find a min-cost tree T* with n leaves, i.e., Cost{T*) = 
min {Cost{T) \ T has n leaves} 



£(T) ~ max{£(u) \ v is a node in T} = max{^(i;) \ v is a leaf in T}. 




(2) 
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Level Depth Signature Cost 




Figure 2: A generalized mixed radix tree with R = (2,2,2,3,3,4) and C (1,1,2,2,1,1). Note that 
n = 12 but to make the tree full we needed to add two extra leavs on the bottom level. Also, the signatures 
and costs on level i are of the truncated tree containing the nodes on the first i levels. Costs are for 
P = {1,2,3,4,5,6,7,8,9, 10, 11,12}. 



2.1 The Basic Top-Down Dynamic Program 

In this section we quickly describe the standard top-down DP formulation. Since variations of this formulation 
have been extensively used before for various coding problems, e.g. [HI [T^l [TUl H] , we only sketch the method 
but do not rigorously prove its correctness. 

In what follows P = {pi,p2, ■ ■ • with pi > P2 > • ■ • ^ ^ is given and fixed. The pi can be 
arbitrary weights and are not required to sum to 1. The pi sequence is implicitly padded so that, for i > n, 
Pi = 0. Finally, for to > 0, set Wm = Y^iymP^- 

We start with some standard simplifying assumptions about min-cost (optimal) trees T* . We will show 
that there always exists at least one optimal tree satisfying these assumptions. Since our goal is to find any 
optimal tree, our search can be restricted to trees satisfying the assumptions. 

In what follows, an internal node w, £(w) = i, in tree T will be full if and only if it has Vi^^i children in T. 
Tree T will be full if all of its internal nodes are full. 

Assumption 1: li i < j then, in T*, depth{vi) < depth{vj). 

If this was not true we could label Vi with pj and Vj with pi . The resulting tree has cost no greater than the 
original one so it remains optimal. We may therefore always implictly assume that leaf weights in trees are 
non-increasing in the level of the tree. 

Assumption 2: There is a full tree T* with the same cost as the optimal tree for n leaves. T* has n' leaves 
where 

max(n, r^rp,-^) < n' < n + r^^x-) ^ 1 
This will be a consequence of the padding of P. 

Let T be an optimal tree with exactly n leaves. Suppose t{T) ~ i{v„) ~ £. 

First note that all internal nodes v with £{v) < £ — 1 are full. Otherwise we could add a new leaf child of 
V at level £{v) 4- 1 < £{vn) and label it with p„, creating a tree with smaller cost than optimal tree T. Thus, 
the only non-full internal nodes in T are on level £ — 1. 

Next note that we may assume that at most one internal node at level (£—1) is not full; otherwise leaves 
on level i can be shifted to the left, so that all internal nodes on level- (i — 1), except possibly the rightmost 
one, are full. Make this rightmost node full by adding an appropriate number leaves to it and call this new 
fuU tree T*. Note that £{T*) = £{T) = £ and cost{T) = cost{T*). This follows because P was padded by 
setting Pi — for i > n. Let n' be the number of leaves in T* . 

By definition n < n' . Since T contains at least one internal node on level i — 1, T* contains at least re 
leaves on level £ — I. Thus max(?i, r{) < n' 

Furthermore, n' < n + 7'f — 1 because T* was created by adding at most rg — \ leaves to T. 
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Assumption 3: t{T*) < n. 

This will follow from the fact that we may assume that Vi,r,; > 2. 

Assumption 4-' The cost of a tree is fully determined by its leaf sequence, i.e., the number of leaves on each 
level. No other structural properties need to be maintained. 

This follows directly from the previous assumptions, i.e., the fullness of optimal trees. Although we will 
talk about constructing trees we will really be constructing the corresponding leaf sequences, e.g., sequences 
denoting how many leaves are on on each level. Since the cost of a tree is fully determined by its leaf sequence 
this does not cause any problems. 

Suppose that T' is a tree with n' > n leaves. Create T" by pruning the deepest leaves from T' one- 
by-one, until exactly n leaves remain. Then, by construction, T" is a tree with exactly n leaves such that 
cost(T") < cost(T'). This observation, Assumption 2 and Assumption 3, tell us that we can find the optimal 
tree for n leaves by first finding ~ for every I < n, and every n' satisfying max(n, ) < n' < n + — 1- the 
cost of the min-cost full tree of level at most t with n' leaves. Then we take the minimum cost tree among all 
such trees and prune it until it has exactly n leaves. The resulting tree wil be the optimal tree for n leaves. 

There are only 0{rP') {£,n') pairs that need to be examined; given their costs, finding the optimal pair 
requires only 0{n^) time. (The subsequent pruning operation can easily be done in 0{n) time.) The hard 
part is, for each given (i, n') pair, to find the costs of the appropriate min-cost full tree and then, if necessary, 
build it. 

The intuition behind the solution is to build optimal trees top-down, starting with an initial tree - the 
root - building successively bigger trees level- by- level by making some nodes on the bottom level internal. 
Part of the specification of these intermediate truncated trees will be an explicit statement of the number of 
nodes on their bottom levels that will become internal when they are further expanded. The process ends 
when a tree whose bottom level contains no internal nodes is constructed. 

Since we are only interested in constructing full trees and the truncation of a full tree up to any level is 
full, this process may implicitly assume that every intermediate tree built is full. 

To transform this intuition into a dynamic program we will need to somehow encode the space of inter- 
mediate trees compactly and introduce an appropriate definition of cost for intermediate trees. 

Definition 4 

Tree T is an i-level tree if all nodes v £ T satisfy £{v) < i. 
If T is an i-level tree its i-level signature is the ordered pair 

sigi{T) (to, &) 

in which 

TO = \{v & T \ V is a leaf, £{v) < i} \ 
b = \ {v ^ T \ V is an internal node £{v) = i} \ 

In the above definition, "internal'' means that the leaf at the bottom level is tagged as being made internal if 
the tree grows to its next level. 

Let T be an i-level tree with sigi{T) — {m,b). The i-level partial cost ofT is 

rn n 

Cost^{T) = depth{vt) ■ pt + L{i) ■ pt (3) 

t=l t=m+l 

Figure [2] illustrates this definition. 

Consider the possible signatures that could occur. Suppose that T is an i-lcvel truncation of some optimal 
tree T* with £{T*) = £ and sig^{T) = (m, b). 

If T is a proper truncation of T*, i.e, & > 0, then £{T) < £. Thus every labelled leaf in T is one of 
vi, . . . ,Vn in T* and every one of the b nodes on the bottom level of T that is labelled as "internal" is the 
ancestor of one of wi, . . . , w„ in T*. Thus m -\- b < n. 
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If T is not a proper truncation of T*, i.e., T = T*, then b = and, from Assumption 2, max(n,r£) < 
m < n + r£ — I. This motivates 

Definition 5 (m, 6) is a valid i-level signature if 

• If b > then m + b < n. 

• 7/5 = then max(n, r^) <TO<n + ri — 1. 

Obviously, the number of valid z-level signatures is O(n^). 
We can now introduce the DP table. 

Definition 6 Let (rn,b) be a valid i-level signature. Set OPT^[m,b] to be the minimum i-level partial cost 
over all i-level trees T with signature {m,b). More precisely 

OPT''[m,b] = min {CostiiT) \ 3T, T is a i-level full tree with sigi{T) = (m, 5)} (4) 

// no such tree exists, set OPT^[m, b] ~ oo. 

li t > n then pt ~ 0. Thus, if T* is an i-level full tree with n' > n leaves then, by definition, Costi{T*) = 
Yl^=idepth{vt) ■ Pt ~ Cost{T*). So, OPr*[n',0] will be the optimal actual cost of an i-levcl full tree with 
n' > n leaves, which is what we want. 

Definition 7 Let T' be an {i — \)-level tree with sigi^i(T') = {m',b'). 
Expand T' to a full i-level tree by adding all rib' nodes on level i. 

For b satisfying < b < b'ri, the b^^ expansion ofT' is the i-level tree created by denoting b of these rib' 
nodes as internal and making the remaining b'ri — b nodes into leaves. We denote this by 

T = Expand.^{T',b). 

Note that sigi(T) = (m, b) where m ^ m' + b'ri — b. 

We now extend the definition of expansions to signatures 

Definition 8 // (m', 6'), (m, 6) are valid signatures such that < b < b'ri and m = m' + b'ri — b we write 

{m' ,b')^{m,b). 

It is easy to prove the following by construction: 

Lemma 1 Let T' be an (j — l)-level tree with sigi{T') ~ (m', b') and (m, b) such that (m', b')'^{m, b). Then 
T = Expandi{T' ,b) is a well-defined i-level tree with sigiiT) ~ {m,b). 

This implies the following corollary, which is the basis of the correctness of the dynamic program. 

Corollary 1 Let {mo, bo) = (0, 1) be the unique level-0 tree with internal root. Then the lemma implies that 
every sequence 

(?7io, bo)^{mi,b2)^ ■ ■ ■ '-H>\?7ii_i, 6j„i) A(mj, bi) (5) 

corresponds to an i-level tree T with sigi(T) — (mi,bi) that can be constructed top-down from the root 
by following the appropriate expansions given by the sequence. In particular, if {mi,bi) = (n',0) then the 
constructed tree has exactly n' leaves. 

Lemma 2 LetT' andT be, respectively, i — 1 and i-level trees with sigi-i{T') ~ {m',b') and sigi{T) = {m,b). 
If {m' ,b')^[m,b) then 

Cost,{T) = Cost,-i{T') + c.W^„'. 



7 



Proof: From Lemma[Tl T = Expand{T\ b) so level i contains the leaves Vm'+i, ■ ■ ■ , ^^?n and 

m 

Costi{T) = y^depth{vt) ■ pt + L{i) ■ ^ pt 

t — 1 ra<.t<.n 



depth{vt) ■ Pt + {L{i - 1) + c,) Pt + 

i — 1 \ rn' <t<m 7n<t<n 



This tells US that the cost of the n'-leaf tree associated with sequence (O can be calculated level by level 
to be ^^tWmt where — n' . Combining all of the above, we can now write a simple DP that models 

building optimal trees from the top-down. 

Lemma 3 The optimal cost of an i-level tree with signature (m, 6) satisfies 

OPT'[m,b]= min (oPT'-^[m',b']+CiWm'}. (6) 

{{m',b') I {m'.b')^{m,b)} ^ 

Initial conditions are that OPT'^[0, 1] = with all other entries being set to oo. 

The entries OPT''[m,b] only depend upon the entries OPT''~^[m,b] so the table can be filled in using 
the order i = 1, 2, . . . , n. 

For any given level i there are only O(n^) valid i-level signatures. From Definition [51 for every (m^b), 

there are only 0{n) signatures such that (m', b')^{m, b) So, for fixed i, filling in all of the OPT'^[m, b] requires 
O(n^) time, with 0(n'*) total time needed to fill in all of the enties OPT'^[m, b], i < n. 

We will now see how, for fixed i, to calculate the values OPT'^[m,b] in only O(n^) time. This will, as 
promised, reduce the total running time for filling in all n levels of the table to to 0{n^). 

2.2 Batching for Speedup 

We now see how to fill in the DP entries in a faster way. We first need two more definitions. 
Definition 9 For 1 < d, define 

I{d) = {(m, b) I m + b = d}, I[{d) = {{m\ b') \ m' + b'r, = d] . 

For any fixed i and (m, b) £ Ii{d), the definition of implies that 

"(m', b')^{m, by if and only if "(m', b') £ Z'(rf) with b < b'r^ . 
This immediately permits rewriting ([6]) as 

Lemma 4 // (m, b) G 1{d) for some d < n and 

OPT'[m,b]= min \oPT'-^[m' ,b'] + c,W„A . (7) 

b<b'ri 

We now claim that, for fixed d < n, the calculation of the values OPT^[m, b] for all (to, b) £ can be 
batched together in in 0{d) = 0{n) time, i.e., in amortized 0(1) time per entry. 

For fixed d < n suppose (to', b') £ T[{d), i.e., to' + b'ri = d. This implies b' < [d/ri\ . Set 

VO < 6' < ld/r,\, 7(6') = OPT-\m', 6'] + c,W^, = OPT'-^[d - b'r,, b'] + c,Wd-r,b. . 

These can be precalculated in 0{d) time. Then ([7]) just says that for (to, b) £ I{d), 

OPT'[m, b] = min |7(6') (b/r,) < b' < [rf/r,J | . (8) 
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(m, b) £ 1{d) implies m = d 



b. Since b < ri\_d/ri\, m > t = d — ri \_d/ri\ = d mod r^. So 



I{d) = {X, 



ra 



m = t, 1} where X^a = {jn^ d — nij . 



Then ([8|) can be rewritten as 



OPT{X,n) = min {7(6') | {d ~ m)/r, < b' < [d/r,\ } 



(9) 



This immediately yields 



OPT[Xt] 



l{[d/r^.\) 

( OPT[X„,_i] if r,; /{d-m) 

\ mm{OPT[X^-i],j{{d-m)/n)) ii n\{d - m) 



Vm > t, OPT[X^] 



Thus, we can calculate all of the OPT[Xm] for Xm G T{d) in 0{d) time by working in the order m = 

For fixed i, to fill in all valid signatures we start by implicitly setting all entries to 00 and then iterate for 
d ~ 1, 2, 3, . . ., for each value of d using 0(d) time to calculate all of the entries OPT^ [m, b] with rn + b = d. 
The question is where to stop the iteration. 

From Definition [5] we know that if 6 > then d — m+b < n while if 6 = then max(n, ri) < m < n+r^ — 1. 
There are now two cases. 

Ti <n : Then m + 6 < 2n so stop the iteration at d = 2n — 1. All of the valid entries will have been filled in 
using Y.d<2nO{d) = O(n^) time. 

ri > n : In this case if 6 = then ri < m < ri + n — 1 and (m', b')^{m, 0) implies that either (m', b') = (m, b) 
or (to', b') = (to — ri, 1), i.e., (to, 0) has only two possible predecessors. We can therefore fill in the full 
table in two phases. In the first, fill in all valid entries OPT^[m, b] with m + b = d for d = 2, 3, . . . , 71 in 
0{n^) time. In the second, fill in the all valid entries of the form OPT^[m, 0] with < m < ri + n — 1 
in 0{n) time, by checking the two predecessors of each possible entry. 

3 Mixed-Radix Coding and Reserved-Length Coding 

We now see how to solve both Mixed-Radix Coding and Reserved-Length Coding via the GMR approach. 

3.1 Mixed-Radix Coding 

Chu and Gill's Mixed-Radix Coding problem |Tl] is exactly the GMR problem restricted to all of the edge 
costs being identically 1, i.e., Vi,Ci — 1. The algorithm in the previous section solves this in 0{n^) time, 
improving upon the 0{n^ logJi) time of jllj . 

3.2 Reserved-Length Coding 

In the reserved-length coding problem, there are restrictions as to permissible codeword lengths. In the tree 
version of the problem, these become restrictions on the allowable levels on which leaves can appear. More 



There are two versions of the problem. 

In the first version of the problem, the given-lengths case, a set of integers A = {71, . . . ,7g} is given 
(w.l.o.g, < 71 < 71 < 72 < • • ■ < 7g; we may also add 70 = since, if n > 1, the root will never be internal) 
and we are asked to find a minimum-cost r-ary tree among all trees with Level(T) C A. 

In the second version of the problem, the g-lengths case, an integer g is given and we are asked to find a 
minimum-cost r-ary tree among all trees with \Level{T)\ < g. 



formally, let 



Level{T) = {e{v) \ v a leaf in T}. 
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Level Depth Signature Cost 




Figure 3: A Reserved-Length tree (left) with A = {1,3,6}, n = 16 leaves and the corresponding GMR tree 
(right). Note that even though they arc allowed, there arc no leaves on level 1. All internal nodes on level 3 
should have 8 descendents on level 6 so 6 leaves were added to make the tree full. Signatures and costs for the 
GMR tree at level i are for the truncated tree containing the first i levels. Costs arc for P = {1, 2, . . . , 16}. 

3.2.1 The Given-Lengths Case 

Let T be an optimal r-ary tree for given P and A = {71, . . . , jg}. 

All leaves in T are at a level % for some 7^ S A. Consider any internal node v at level 7i-i. It has no leaf 
descendants at any level £ with 7i_i < £ < 7,;. We may therefore assume that all of its r'^^'^^-'^ descendants 
at level 7^ are in the tree, i.e., that v is the root of a complete subtree of height 7; — 7^-1. 

We may therefore create a new tree T' as follows. The root of T' corresponds to the root of T. Nodes in 
T' at level i are in 1-1 correspondence with nodes at level 7^ in T and there is an edge from node u on level 
i — 1 to node v on level i in T' if u is the level 7i_i ancestor of v in T. Sec Figure [3] for an illustration. 

By construction, T' is a GMR tree with — r'^i~'ii-'^ and c; ~ 7^ —7^-1. Furtheremorc, the construction 
can be reversed, with any generalized mixed arity tree with these parameters being transformable into a 
restricted length tree with the same cost for the given A. 

Since there arc at most g levels, our generic GMR algorithm solves this problem in 0{gn?) time, improving 
upon the 0{gn^) algorithm of [4] . 

3.2.2 The 5-lengths case 

If the levels on which leaves appeared were known to be A = {71 , . . . , 7^} then this is exactly the given-lengths 
case, which as seen, is equivalent to building an optimal GMR tree with {ri,Ci) = (r'*"'"'''"^ , 7^ —7^-1). The 
added complication here is to guess A. 

This is equivalent to the problem of building a slightly generalized version of a GMR tree in which, 
instead of guessing A. we instead, at each level i, guess the pair (ri,Ci) = (r*,t) for t > 1. Any such pair is 
allowable but once t is chosen, it applies to all nodes on level i. Furthermore, since the tree only needs n 
leaves we may assume r* < rn and thus may restrict t < 1 + log^ n. 

This motivates slightly modifying the GMR model to allow choices of (r,, q). 

Recall that the original definition of GMR specifies arities R = (ri,r2,...) and edge lengths C = 
(ci, C2, . . .). We now replace these with R = (ri, r2, . . .) and edge lengths C = (ci, C2, . . .) where 

ri = {n^i, ri^2, ■ ■ ■ ,ri^A,}, Ci = {ci^i, q,2, ■ • ■ ,Ci,Ai}, 

are sets of Aj possibilities for level i. A permissible tree is a GMR tree for some sequence R = {rij-^ , ^2^2 , ■ • ■) 
and edge lengths C — (ci , C2J2 1 ■ • ■) where Vi, 1 < ji < A^. 

Given P, an optimal tree would now be a min-cost permissible tree for the given i?, C. 

The discussion above tells us that to solve the g-lengths problem, it is only necessary is solve this new 
generalized version of the GMR problem to construct a minimum-cost g-level tree where, for every i, ri and 
Ci are the sets defined by 

A, = 2 + [logv, n\ , and VI < j < A,, r^^ = r^-\ c.j = j - 1. 
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The modifications to the definitions and algorithms are straightforward. Signatures are defined the same 
way as before. Definition [5] of vahd signatures needs to be modified to aUow rij . 

Definition 10 (m, b) is a vahd i-level signature if 

• If b > then m + b < n. 

• 7/5 = then 3j, such that max(n,rij) < m < n + ri_j — 1 

Note that the number of vahd z-level signatures is 0{n^ + nAi). 
Definition [8] also needs to be slightly generalized: 

Definition 11 // (m', fe'), (m, b) are, respectively, valid (i — 1) and i-level signatures such that < b < b'ri,j 
and m = m' + b'rij — b, we write 

(m'j b)-^{m^ b). 

We now, similarly as before, define 

OPT^[m,b] = min{Costi{T) \ 3T, T is a i-level tree with sigi{T) = (m, 5)} (10) 

The only major difference is in the analogue of Lemma[31 which gives the DP for calculating OPT* [to, b]. 
This now needs to be split into two phases; the first calculates, for every j , the optimimum value of OPT^ [m, b] 
assuming that (r.^, Ci) = {rij, Cij). The second takes the minimum of this value over all j. More specifically: 

Lemma 5 

Forl<j <Ai^i, set OPT''^m,b]= min loPT'-^[m' ,b'] + a^jW^n'} ■ (H) 

{(m',6') I {m',b')'^{m,b)} 

The optimal cost of an i-level tree with signature (to, 6) then satisfies 

OPT'\m,b]= min OPr*'^[m,W (12) 

l<i<A, 

Initial conditions are that OPT^iO, 1) = with all other entries being set to oo. 

Given the values OPT'~^[to, 6], the batching speedup of Subsection 12.21 now permits, for any fixed j, 
calculating all of the values OPT^'^[m, b] in 0{n^) time. Thus, all of the values OPT^[m, b] can be calculated 
in O(Ajn^) time. 

The total amount of work required for calculating OPT^[m, b] from scratch is then O (n^ Yli=o ^0' 
In the g-lengths problem, = 0(log^ n) so the total runnnig time for solving the g-lengths problem is 
0{gn^ logn), improving the 0{n^g^ log^""'^ n) running time of the algorithm in [4]. 

4 One-Ended Coding 

We now consider the problem of constructing minimum-cost binary prefix-free codes having the property 
that each codeword ends with a "1" . The original algorithms [6l [9] for this problem were exponential. 
[lOj presented a top-down DP running in 0{n^) time. Using the batched speedup technique developed in 
Subsection 12 . 21 we can develop a modified top-down DP that reduces the running time to O(n^) 

As in Section [2l the algorithm will find a min-cost coding tree. We must first modify the code-tree 
correspondence to reflect the new 1-ended requirement. Assume that a left edge is labelled with a '0' and a 
right edge with a '1'. A node is a 0-nodc (1-node) if the edge connecting it to its parent is labelled by a 
(1). Wc will extend this naturally to 0-lcavcs and 1-lcavcs, and 0-intcrnal nodes and 1-internal nodes. 

To reflect the 1-ended restriction on the codes, only 1 leaves will be labelled with probabilities from P = 
{pi, . . . ,Pn}- Let Vi be the 1-lcaf in tree T labelled with pi. Then wc may still write CostiT) = d{vi)pi. 
As before, wc pad P so that \i i > n then pi — 0. 

These changes require that we naturally modify the definition of full-trees, signatures and expansions. 
Doing this yields a DP with a size 0{n^) OP[m, b] table. The naive algorithm for filling in this table would 
require 0{n^) time but a batching argument very similar to that in Subsection 12. 21 permits filling in the table 
in 0{n^) time. The step by step modifications required to change the GMR algroithm into one for 1-ended 
coding are given in the Appendix. 
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A The One-Ended Coding Algorithm 



Wc modify the simplifying assmiiptions on optimal trees T* of Subsection l2.1l to reflect the extra requirement 
of being 1-ended. Note that not every optimal tree will satisfy these assumptions but we will show that at 
least one optimal tree will. These assumptions will therefore permit us to restrict the space of trees in which 
to search. Figure S] illustrates the concepts and definitions introduced here. 

Assumption 1: li i < j then, in T*, depth(vi) < depth{vj). 
This is the same as before 

Assumption 2: Let £ = i{T*). 

(a) T* is full. 

(b) All 1-internals are on levels < i — 1. 

(c) The only 0-leaves are on level ^. 

(d) There will be at most n — 1 1-leaves in T* on levels < £ 

(e) The number of 0- internals on level £ — 1 will be < n — 1 . 

Let T be an optimal 1-ended tree with exactly n 1-leaves. Erase all 0-nodes that do not have a 1-leaf 
descendant and then make the tree full by adding appropriate missing edges so that every internal node has 
two children. Note that after doing this, the 1-sibling of every 0-leaf exists in T as either an internal node 
or one of the n 1-leaves. 

Let Vn be the node labelled with p„. We may assume that = £; otherwise we could erase all nodes 

on level £ and get a smaller full tree with the same cost which could replace T. 

Suppose w is a 1-internal node that has fewer than two 1-leaf descendents. Since T is full v must have 
exactly one 1-leaf descendent. Erase the subtree rooted at v. The resulting tree would still have n 1-leaves but 
a smaller cost, contradicting the optimality of T. Thus every 1-internal has at least two 1-leaf descendents. 
This immediately implies that all 1-internals have subtrees of height at least 2 hanging off of them, so all 
1-internal nodes are on levels < £ — 1. 

If u is a 0-leaf then £{u) > i — 1; otherwise, make u the parent of a 1-leaf at level £{u) + 1 < £ and move 
Pn to this 1-leaf. This reduces the cost of the tree, contradicting the optimality of T. 

Thus, all 0-leaves in T' are on level £ — 1 or £. Let u be any 0-leaf on level £ — 1. Since T is full, its 
1-sibling V is also in T, Since £{v) = £ — 1, we have already seen that d is a 1-leaf. 

By assumption, since T has exactly n 1-leaves and one of them is at level £, the number of 1-leaves on 
levels < £ is at most n — 1. In particular, since every 0-leaf on level £ ~ 1 has a 1-leaf sibling, there are most 
n — 1 such 0-leaves. 

Now create T* from T by making every 0-leaf on level £—1 internal. This adds at most n—1 new 1-leaves 
to the tree; all of the new 1-leaves added will be at level £ and be labelled with the padded Os, so this does 
not change the cost. Since T is optimal for n 1-leaves, T* is optimal for the number of 1-leaves it has. 

The T* thus created satisfies the assumption. 

The remaining two assumptions will be the same as in the GMR case. 

Assumption 3: £{T*) < n. 

Assumption 4-' The optimality of a tree is fully determined by its leaf sequence, i.e., the number of leaves on 
each level. No other structural properties need to be maintained. 

Our characterization of nodes will be slightly different than in the GMR case. A node in T* will be a 
good if it's a 1-leaf that gets labelled with one of pi, . . . ,p„. It will be bad if it is a 0-node or a 1-node that 
doesn't get lablelled with one of pi, . . . ,p„. 

We will build trees T* from the top-down, level- by-level. At step i our current tree T will be the first i 
levels of T* . We will identify in T the the number of nodes that are good leaves and the number of nodes, 
on T's bottom level, that are bad. Note that if i < £{T*) then, in the next expansion of level, all bad nodes 
on the current bottom level will become internal, each one contributing one new 1-node and one new 0-node 
on the new bottom level. If i = £{T*) then T = T* and the bad nodes are extra (those that get labelled 
with pj, j > n) 1-leaves and 0-leaves on level i. 

This motivates us to change the definition of signatures and cost as follows: 
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Level (also Depth) Signature Cost 




Figure 4: A One-Ended Coding tree for ji — 8 leaves. Note that only 1-nodes are labelled leaves. On the 
bottom level, the two dotted 0-leaves and their two grey 1-leaf siblings are 'bad' nodes added to make the 
tree full. Signatures and Costs at level i are those of the (truncated) subtree containing the first i levels of 
the tree. Costs are calculated for P — {1, 2, 3, 4, 5, 6, 7, 8}. 



Definition 12 

If T is an i-level tree its i-level signature is the ordered pair 

sigi{T) ^ {m,b) 

in which 

m = \ {v T \ V is a good 1-leaf, i{v) < i} \ 
b ~ \ {v £ T \ V is a bad node, £{v) — i} 

In the above definition, b is counting leaves in T that, if the tree is expanded one level further, will become 
internal nodes. Let T be the starting (0-level) tree containing only the root. Since the root will always be 
expanded, it is bad, so sig(){T) = (0, 1). This will later be the starting point of our dynamic program. 
Let T be an i-level tree with sigi{T) = (rn,b). The i-level partial cost ofT is 

rn n 

Costi{T) = ^ depth{vt) -pt + i- ^ pt (13) 

t=l t=m+l 

As in the GMR case, we want to build an optimal tree T* that satisfies the assumptions. We start by 
noting that, by assumptions 2(b) and 2(c), the only bad leaves arc on the bottom level i = £{T*). 

The parents of these bad leaves are on level £ ~ 1 and therefore must be 0-nodes, since by assumption 
2(b), all 1-nodes on level £ — I are leaves. From assumption 2(e) there are at most n — I such 0-internals. 
So, level £ contains at most n — 1 bad 1-leaves and n — 1 bad 0-leaves. 

Now suppose that T* is being built top-down level- by-level. Let T be the first i levels of T* with 
sigi{T) = (m, b). By definition m < n. Consider the bad nodes on the bottom level of T. li i < £ then every 
bad node is internal. From the fullness of the tree, every bad node on level i must have a 1-leaf descendent 
in T* . There are at most 2n — 1 (n good and n — 1 bad) 1-leaves in T and each one can appear at most once 
in some subtree rooted at level i so 6 < 2n — 1. If i = £ then b is the number of bad leaves in T* which is 
< 2(n — 1). We may therefore assume that b < 2n — 1. 

This motivates an analogue of Definition [SJ 

Definition 13 (m, b) is a valid signature if 

m < n and b < 2n ~ 1 
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We also modify Definition [HI 

Definition 14 Let (m, b) be a valid signature. Set OPT[m, b] to be the minimum i-level partial cost over 
all i and all i-level trees T with signature {m,b). More precisely 

OPT[m,b] = min {Costi{T) \ 3i,T, T is a i-level tree with sigiiT) = (m,5)} (14) 

// no such tree exists, set OPT[m, b] ~ oo. 

Note tliat if m — n, tlicn OPT[n, b] = X^iLi depth{vt) ■ pt- Thus, from, our previous discussion 

mm{OPT'[n, b] \ 1 < 6 < 2n - 2} 

is the cost of the solution to the one-ended coding problem. 

Now suppose that T' is an {i — l)-level tree with sigi-i = (m', b'). If T' is expanded one more level then 
the b' bad nodes on level i — 1 become internal with each one contributing one 1-node and one 0-node to 
level i. Between and min(&', n — m') of the 1-nodes can become good 1-leaves. The remaining 1-nodes and 
all b' 0- nodes are bad. The newly created i-level tree has sigi{T) = (to, b) where 

m' <m< n, m = m' + 2b' - b, I < b' < b < 2b'. (15) 

We therefore define expansions as follows: 

Definition 15 If [m! ,b'), {m,b) are valid signatures that satisfy il5\) we write 

(to', 6) (m, b). 

The next few steps exactly follow the development of Lemmas[Tl[21[3]and Corollary[l]with almost identical 
proofs, so we do not state them explicitly here. The final result is that 

OPT\m,b]= min \ OPT'~^\m' ,b'] + W,„,] . (16) 

{{m'M)\{ni',b')^(m,b)}l ^ ' J 

with initial condition OPT^(Q, 1) = 0. Note that (rn',b') (rn,b) implies either m' < m or m — m and 
b = 2b'. So if {m',b') {iTi,b) then {m',b') is lexicographically smaller than {m,b). Thus, the table can 
be filled in in lexicographic order to correctly calculate the final values. After filling in the table, the final 
solution value will be the minimum value of OPT[n, 6], where 1 < 6 < 2n — 2. The actual tree corresponding 
to this solution can be built by backtracking through the table to find the sequence of expansions 

(too,6o) (toi,62) (toj_i,6j_i) -> (n, 6) (17) 

that corresponds to a min-cost one-ended tree. 

The DP table has 0{n^) entries and each entry requires 0{n) time to calculate, leading to an 0{n^) 
algorithm for constructing the optimal tree. An algorithm with this nmning time (based upon a slightly 
different DP) was given in [TU] . 

We now show that it is possible, using the batching technique earlier introduced for the GMR, to fill in 
this table in O(n^) time. Define 

Definition 16 For \ < d, define 

I{d) = {(to, 6) I m + b^d, < b, m < n} , 
I'{d) = {(to', 6') I m' + 2b' = d, < 6, m < n} . 

For any fixed d (to, b) G X(rf), the definition of implies that 

"(to', b') (to, by if and only if "(to', b') £ T [d) with 6/2 < 6' < 6". 
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Note that if {m,b) £ I'{d) then m' + b' < d so {m',b') G T{d') with d' < d. Also, since m < n and 
6 < 2n — 1, we can bound d < 3n — 1. We may therefore fill in the DP table by filling in all the entries in 
T{d) as a batch in the order d = 2, 3, 4, . . . 3n — 1. 

We now see that for fixed d we can, by batching, fill in all of the entries in 2{d) in 0{d) time. This will 
allow filling in the entire table in O(n^) time, delivering the promised time reduction. 

For fixed d suppose (m', b') S T{d), i.e., m' + 25' = d. Since < m' and < 6' we have 1 < 6' < d/2. 
For such b' set 



Note that since m' + b' < d, the values OPT[m', b'] have already been calculated and can therefore be looked 
up. Therefore, all of the j{b') can be calculated using a total of 0{d) time. Then, for (m, 6) S I{d), 



Stated this way. this is just a special case of the Range Minimum Query (RMQ) problem. 

Given an array A of size n, the RMQ problem is to construct a data-structure that, given two indices 
i < will return the index of a smallest valued item in the subarray A[i] . . . A[j]. There are known algorithms, 
G-g-, [mis], for (9(??) construction of a data-structure that permits 0(1) RMQ queries. In our case, the array 
is the b' indexed set of jib') values. The array has size 0{d) and its values can be constructed in 0{d) time. 
[14i [5| then permit us, in 0{d) time, to construct an RMQ data structure. Once this structure is constructed, 
each value OPT[m, b] can be found in 0(1) time. Thus, as promised, all of the OPT[m, b] for (m, b) G I{d) 
can be batched together and evaluated in 0{d) time 



7(6') = OPT[m', b'] + W„,' 



OPT[d^2b',b'] + Wd-2b'- 




(18) 
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